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Abstract — This  paper  examines  the  efficacy  of  a  novel  method 
for  anomaly  detection  in  mechanical  systems,  which  makes  use 
of  a  hidden  Markov  model,  derived  from  the  time-series  data  of 
pertinent  measurement(s).  The  core  concept  of  the  anomaly  de¬ 
tection  method  is  symbolic  time-series  analysis  that  is  built  upon 
the  principles  of  Automata  Theory,  Information  Theory,  and  Pat¬ 
tern  Recognition.  The  performance  of  this  method  is  compared 
with  that  of  other  existing  pattern-recognition  techniques  from 
the  perspective  of  early  detection  of  small  fatigue  cracks  in  duc¬ 
tile  alloy  structures.  The  experimental  apparatus,  on  which  the 
anomaly  detection  method  is  tested,  is  a  multi-degree-of-freedom 
mass-beam  structure  excited  by  oscillatory  motion  of  two  electro¬ 
magnetic  shakers.  The  evolution  of  fatigue  crack  damage  at  one  or 
more  failure  sites  are  detected  from  symbolic  time-series  analysis 
of  displacement  sensor  signals. 

Index  Terms — Anomaly  detection,  fatigue  crack  damage,  sym¬ 
bolic  dynamics,  time-series  analysis. 

I.  Introduction 

AN  ANOMALY  is  defined  as  deviation  from  the  nominal 
behavior  of  a  dynamical  system  and  is  often  associated 
with  parametric  and  nonparametric  changes  that  may  gradually 
evolve  in  time.  Anomalies  may  manifest  themselves  with  self¬ 
excitation  within  the  dynamical  system,  or  under  excitation  of 
exogenous  stimuli.  On  the  sole  basis  of  the  fundamental  prin¬ 
ciples  of  physics,  the  accurate  and  computationally  tractable 
modeling  of  complex  system  dynamics  is  often  infeasible;  such 
a  model  may  only  exhibit  the  ensemble-averaged  behavior  of 
the  physical  process  [1],  Therefore,  it  is  necessary  to  rely  on 
time-series  data  generated  from  sensors  and  other  sources  of 
information  [2], 

Several  researchers  [3] — [5]  have  addressed  the  problem  of 
structural  health  monitoring  based  on  time-series  data.  Along 
this  line,  a  novel  concept  of  anomaly  detection  has  been  pro¬ 
posed  by  Ray  [6],  where  the  underlying  information  on  the 
dynamical  behavior  of  complex  systems  is  derived  based  on  the 
following  assumptions: 

1)  the  process  has  stationary  dynamics  at  the  fast  time  scale; 
2)  any  observable  nonstationary  behavior  is  associated  with 
changes  evolving  at  the  slow  time  scale  at  which  anomalies 
may  occur. 
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The  above-mentioned  method  of  anomaly  detection  requires 
both  temporal  and  spatial  discretization  of  the  pertinent  mea¬ 
surement  data  to  construct  a  sequence  of  symbols  [7],  [8],  The 
symbol  sequence  is  treated  as  a  transform  of  the  original  time- 
series  data  of  measurements  from  the  phase  space  into  a  sym¬ 
bol  space  such  that  no  significant  information  is  lost  [9],  [10]. 
This  transform  method  is  called  symbolic  time-series  analysis 
in  technical  literature  [2], 

This  paper  makes  use  of  symbolic  time-series  analysis  for 
early  detection  of  small  anomalies,  resulting  from  fatigue  crack 
damage  in  ductile  alloys,  which  is  a  major  source  of  failures  in 
structural  components  of  operating  machinery  [11].  The  objec¬ 
tive  is  to  capture  this  information  from  the  observed  time-series 
data  as  early  as  possible  so  that  the  decision  and  control  system 
may  take  appropriate  actions  to  avert  catastrophic  failures  and 
satisfy  the  mission  requirements  albeit  at  a  degraded  level  of 
performance  [12]. 

The  anomaly  detection  method  is  experimentally  validated 
on  a  laboratory-scale  test  apparatus,  where  the  source  of  possi¬ 
ble  anomalies  is  fatigue  damage  in  vibrating  mechanical  struc¬ 
tures.  (Note  that  the  evolution  of  fatigue  damage  is  several  or¬ 
ders  of  magnitude  slower  than  the  structural  dynamics.)  The 
test  apparatus  is  designed  to  have  sufficient  complexity  in  it¬ 
self  due  to  partially  correlated  interactions  amongst  its  indi¬ 
vidual  components  and  functional  modules  [1],  Performance 
and  efficacy  of  the  proposed  anomaly  detection  method  has 
been  assessed  for  detection  of  slowly  evolving  fatigue  dam¬ 
age  in  the  short-crack  regime  by  comparison  with  other  ex¬ 
isting  pattern  recognition  techniques,  such  as  principal  com¬ 
ponent  analysis  (PCA),  multilayer  perceptron  neural  networks 
(MLPNN),  and  radial  basis  function  neural  networks  (RBFNN) 
[  13]— [15], 

The  paper  is  organized  into  eight  sections  including  the  pre- 
sentone.  Section  II  briefly  describes  the  test  apparatus  for  exper¬ 
imental  validation  of  anomaly  detection.  Section  III  discusses 
how  fatigue  crack  damage  evolves  as  an  anomaly  in  the  dy¬ 
namical  behavior  of  the  test  apparatus.  Section  IV  briefly  re¬ 
views  the  basic  concepts  of  symbolic  dynamics  for  anomaly 
detection  in  complex  systems.  Section  V  introduces  the  D- 
Markov  machine  method  [6]  for  anomaly  detection.  Section  VI 
presents  existing  pattern-recognition  techniques  that  have  rele¬ 
vance  in  the  present  context  of  anomaly  detection.  Section  VII 
presents  the  experimental  results  and  compares  the  D-Markov 
machine  method  with  other  existing  pattern-recognition  tech¬ 
niques  from  the  perspective  of  early  detection  of  small  fatigue 
cracks  in  ductile  alloys.  Finally,  the  paper  is  summarized  and 
concluded  in  Section  VIII  with  recommendations  for  future 
research. 
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Fig.  1.  Schematic  diagram  for  the  test  apparatus. 


TABLE  I 

Structural  Dimensions  of  the  Test  Apparatus 


Component 

Material 

Length  (mm), Mass  (kg) 

(Length  x  Width  x  Thick¬ 
ness) 

Mass  1 
Mass  2 
Mass  3 
Beam  1 
Beam  2 
Specimens 

Mild  Steel 
Aluminium  6061-T6 
Mild  Steel 

Mild  Steel 
Aluminium  6061-T6 
Aluminium  6061-T6 

2.82 

0.615 

3.87 

800  x  22  x  1 1 

711.2  x  22.2  x  11.1 

203.2  x  22.2  x  11.1 

II.  Description  of  the  Test  Apparatus 

The  test  apparatus  is  designed  and  fabricated  as  a  multi- 
degree-of-freedom  (DOF)  mass-beam  structure  excited  by 
oscillatory  motion  of  two  vibrators  as  shown  in  Fig.  1 .  Physical 
dimensions  of  the  pertinent  components  are  listed  in  Table  I. 
Two  of  the  three  major  DOFs  are  directly  controlled  by  the  two 
actuators,  Shaker  #1  and  Shaker  #2,  and  the  remaining  DOF  is 
observable  via  displacement  measurements  of  the  three  vibrat¬ 
ing  masses:  Mass  #1,  Mass  #2,  and  Mass  #3.  The  inputs  to  the 
multivariable  mechanical  structure  are  the  forces  exerted  by  the 
two  actuators;  and  the  outputs  to  be  controlled  are  the  displace¬ 
ments  of  Mass  #2  and  Mass  #3.  The  failure  site  in  each  specimen, 
attached  to  the  respective  mass  is  a  circular  hole  (of  radius  3.81 
mm)  as  shown  in  Fig.  1 .  The  test  apparatus  system  is  logically 
partitioned  into  two  subsystems:  1)  the  plant  subsystem  consist¬ 
ing  of  the  mechanical  structure  including  the  test  specimens  to 
undergo  fatigue  crack  damage,  actuators  and  sensors  and  2)  the 
instrumentation  and  control  subsystem  consisting  of  computers, 
data  acquisition  and  processing,  and  communications  hardware 
and  software.  Frequency  of  the  reference  signal  is  11.39  Hz 
that  is  the  resonating  frequency  associated  with  Mass  #3  in  the 
mechanical  structure.  The  test  specimens  are  thus  excited  by 
different  levels  of  cyclic  stress  as  two  of  the  specimens  are  di¬ 
rectly  affected  by  the  vibratory  inputs  while  the  remaining  one 
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is  subjected  to  resulting  stresses,  thus  functioning  as  a  coupling 
between  the  two  vibrating  systems.  In  the  present  configura¬ 
tion,  three  test  specimens  are  identically  manufactured  and  their 
material  is  6061-T6  aluminum  alloy.  Nevertheless,  different  ma¬ 
terials  can  be  selected  for  individual  specimens  that  may  also 
undergo  different  manufacturing  procedures.  Future  research 
will  allow  different  materials  and  manufacturing  methods  for 
individual  specimens. 


III.  Generation  of  Fatigue  Crack  Anomaly 

The  mechanical  system  of  the  test  apparatus  in  Fig.  1  is  per¬ 
sistently  excited  near  resonance  so  as  to  induce  a  stress  level  that 
causes  fatigue  failure  to  yield  an  average  life  of  approximately 
20  000  cycles  having  a  total  duration  of  about  32  min.  There  ex¬ 
ists  considerable  scatter  in  the  fatigue  data,  and  variations  have 
been  seen  in  the  actual  observed  life  of  the  specimens  tested  at 
an  identical  stress  level.  The  scatter  results  as  a  consequence  of 
fatigue  sensitivity  to  a  number  of  test  and  material  parameters 
including  specimen  fabrication  and  surface  preparation,  metal¬ 
lurgical  variables,  specimen  alignment  in  the  apparatus,  mean 
stress,  and  test  frequency  [16].  These  uncertainty  factors  were 
taken  into  consideration  for  design  of  the  three  failure  sites  as 
shown  in  Fig.  1. 

The  dynamical  system  attains  stationary  behavior,  in  the  fast 
time  scale  of  machine  vibrations,  under  persistent  excitation 
in  the  vicinity  of  the  resonant  frequency.  The  applied  stress  is 
dominantly  flexural  (i.e.,  bending)  in  nature  and  the  amplitude  of 
oscillations  is  symmetrical  about  the  zero  mean  level,  i.e.,  it  is  a 
reversed  stress  cycle  [17],  Under  such  cyclic  loading  conditions, 
the  specimens  undergo  fatigue  cracking  where  the  gross  stress 
is  elastic  and  plasticity  is  only  localized.  The  fatigue  damage 
occurs  at  a  time  scale  that  is  (several  order  of  magnitude)  slow 
relative  to  the  fast  time  scale  dynamics  of  the  vibratory  motion 
and  eventually  leads  to  a  catastrophic  failure.  Close  observation 
indicates  that  fatigue  failure  develops  in  the  following  pattern: 
1)  the  repeated  cyclic  stress  causes  incremental  crystallographic 
slip  and  formation  of  persistent  slip  bands;  2)  gradual  reduction 
of  ductility  in  the  strain-hardened  areas  results  in  the  formation 
of  submicroscopic  cracks;  and  3)  the  notch  effect  of  the  submi- 
croscopic  cracks  concentrates  stresses  until  complete  fracture 
occurs.  Crack  initiation  may  occur  at  a  microscopic  inclusion  or 
at  local  site(s)  of  local  stress  concentration.  In  this  experimental 
apparatus,  the  sites  of  stress  concentration  are  localized  by  creat¬ 
ing  a  hole  in  each  of  the  three  specimens.  Since  the  mechanical 
structure  of  the  test  apparatus  consists  of  beams  and  masses, 
the  underlying  dynamics  can  be  approximated  by  a  finite  set 
of  first-order  coupled  differential  equations  with  parameters  of 
damping  and  stiffness.  The  damping  coefficients  are  very  small 
and  the  stiffness  constants  very  slowly  change  due  to  the  evolv¬ 
ing  fatigue  crack.  The  main  objective  of  the  work  reported  in  this 
paper  is  to  detect  the  slowly  evolving  anomaly  (i.e.,  decrease 
in  stiffness)  at  an  early  stage  by  observing  time-series  data  of 
the  available  displacement  measuring  sensors.  (Additional  ul¬ 
trasonic  sensors  will  be  installed  at  local  crack  sites  in  future  for 
enhancement  of  anomaly  detection.) 
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Fig.  2.  Conceptual  view  of  symbolic  time-series  analysis. 


IV.  Symbolic  Dynamics  and  Encoding 

The  concept  of  Symbolic  Dynamics  and  its  usage  for  encoding 
nonlinear  system  dynamics  from  the  observed  time-series  data 
have  been  reported  in  literature  [2],  [6].  It  serves  as  a  tool  for 
behavior  description  of  nonlinear  dynamical  systems  based  on 
the  concept  of  formal  languages  for  transitions  from  smooth 
dynamics  to  a  discrete  symbolic  description  [1],  [6]. 

Fig.  2  elucidates  partitioning  of  a  compact  (i.e.,  closed  and 
bounded)  region  of  the  phase  space  and  a  mapping  from  the 
partitioned  space  into  the  symbol  alphabet,  which  becomes  a 
representation  of  the  system  dynamics  defined  by  the  trajecto¬ 
ries.  It  also  shows  the  construction  of  a  hidden  Markov  model 
(HMM)  from  the  symbol  sequence  as  a  finite-state  machine.  The 
quasi-stationary  probability  histograms  of  the  states  represent 
patterns  that  are  indicative  of  the  nominal  (or  reference)  and 
anomalous  behavior  of  the  dynamical  system,  as  explained  in 
later  sections. 

Several  methods  of  phase-space  partitioning  have  been  sug¬ 
gested  in  the  literature  [1],  [18],  [19].  One  such  method  is  sym¬ 
bolic  false  nearest  neighbors  (SFNN)  [9]  that  finds  a  “gen¬ 
erating”  partition  for  symbolic  orbits  to  uniquely  identify  a 
continuous-space  orbit;  thus,  the  symbolic  dynamics  become 
equivalent  to  the  continuous-space  dynamics.  The  key  criterion 
for  SFNN  partitioning  is  that  short  sequences  of  consecutive 
symbols  ought  to  localize  the  corresponding  phase-space  points 
as  closely  as  possible.  This  is  achieved  by  forming  a  particu¬ 
lar  geometrical  embedding  of  the  symbolic  sequence  under  the 
candidate  partition  and  minimizing  the  apparent  errors  in  local¬ 
izing  phase-space  points.  The  nearest  neighbor  to  each  point 
in  the  embedding  is  found  in  terms  of  Euclidean  distance  of 
symbolic  neighbors.  In  general,  better  partitions  yield  a  smaller 
proportion  of  SFNN.  For  convenience  of  implementation,  the 
partitions  are  parameterized  with  a  relatively  small  number  of 
free  parameters.  This  is  accomplished  by  defining  the  partitions 
with  respect  to  a  set  of  radial-basis  “influence”  functions.  The 
statistic  for  SFNN  is  minimized  over  these  free  parameters  using 
genetic  algorithms  [9].  However,  this  partitioning  method  may 


become  computationally  very  inefficient  when  the  dimension  of 
the  phase  space  is  large  or  if  the  data  set  is  contaminated  by 
noise. 

This  paper  has  adopted  an  alternative  partitioning  approach 
for  construction  of  symbol  sequences  from  the  time-series 
data,  which  is  particularly  effective  for  noisy  data  from  high¬ 
dimensional  dynamical  systems.  In  this  method,  called  wavelet 
space  (WS)  partitioning  [6],  [10],  the  time-series  data  are  first 
converted  to  the  wavelet  transform  data  at  different  scales  and 
time  shifts.  The  graphs  of  wavelet  coefficients  versus  scale,  at 
selected  time  shifts,  are  stacked  starting  with  the  smallest  value 
of  scale  and  ending  with  its  largest  value  and  then  back  from  the 
largest  value  to  the  smallest  value  of  the  scale  at  the  next  instant 
of  time  shift.  The  arrangement  of  the  resulting  scale  series  data 
in  the  WS  is  then  partitioned  with  alphabet  size  |.4|  by  horizontal 
lines  such  that  the  regions  with  more  information  are  partitioned 
finer  and  those  with  sparse  information  are  partitioned  coarser. 
In  this  approach,  the  maximum  entropy  is  achieved  by  the  par¬ 
tition  that  induces  uniform  probability  distribution  of  the  sym¬ 
bols  in  the  symbol  alphabet.  Uniform  probability  distribution  is 
a  consequence  of  the  maximum  entropy  partitioning. 

V.  D-Markov  Machine  Model 

This  section  introduces  the  D-Markov  machine  model  [6]  for 
representing  patterns  in  a  symbolic  process,  which  is  motivated 
from  the  perspective  of  anomaly  detection  and  is  an  alternative  to 
the  e-machine  [20].  The  core  concept  of  the  D-Markov  machine 
is  succinctly  presented  below. 

Fet  the  symbolic  representation  of  a  discrete-time, 
discrete-valued  stochastic  process  be  denoted  by:  S  = 
. . .  S-2S-1S0S1S2  ....  At  any  instant  f,  this  sequence  of  ran¬ 
dom  variables  can  be  split  into  a  sequence  S  t  of  the  past  and 
a  sequence  of  the  future.  A  s  s  u  m  i  ngco  n  d  i  t  i  o  n  a  I  stationarity 
of  the  symbolic  process  S  (i.e.,  P[ |  S  t  =  ts"]  being  indepen¬ 
dent  of  t),  the  subscript  t  can  be  dropped  to  denote  the  past  and 
future  sequences  as  S  and  ,  respectively.  A  symbol  string, 
made  of  the  first  L  symbols  of  A'\  is  denoted  by  S  .  Similarly, 
a  symbol  string,  made  of  the  last  L  symbols  of  S ,  is  denoted 
by 

For  D  £  N,  the  set  of  positive  integers,  a  stochastic  symbolic 
stationary  process  is  called  /All-order  Markov  process  if  the 
probability  of  the  next  symbol  depends  only  on  the  previous  D 
symbols,  i.e.,  the  following  condition  holds: 

P(s.;|Sj  — IS;  — 2...)  =  P(Si  |Sj  — i  .  .  .  Si— o').  (1) 

Alternatively,  symbol  strings  *S  ,*S  €  Is  become  indistin¬ 
guishable  whenever  the  respective  substrings  *S  and  *S  , 
made  of  the  most  recent  D  symbols,  are  identical.  Thus,  a  set 
{  A7  :  L  >  ])  }  of  symbol  stings  can  be  partitioned  into  a  max¬ 
imum  of  \A\D  equivalence  classes  [6],  where  A  is  the  sym¬ 
bol  alphabet.  Each  symbol  string  in  {*S  :  L  >  1) } ,  derived 

from  a  stationary  process,  belongs  to  exactly  one  of  the  \A\D 
equivalence  classes.  Given  D  £  N  and  a  symbol  string  V  with 
Its"!  =  D ,  the  effective  state  q(D,iW)  is  the  equivalence  class 
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of  symbol  strings  defined  as 

q(D,  V)  =  {*S  G  "S  :  =  <s}  (2) 

and  the  set  Q (D)  of  effective  states  of  the  symbolic  process  is 
the  collection  of  all  such  equivalence  classes.  That  is 

Q(D)  =  {q(D,*s):*s  (3) 

and  hence  |Q(D)  |  =  \A\D .  A  random  variable  for  a  state  in  the 
above  set  Q  of  states  is  denoted  by  Q  and  the  jth  state  as  qj. 
The  probability  of  transitions  from  state  qj  to  state  qk  is  defined 
as 

TTjk  =  P(s  G  qj  G  Q ,(s,qj)  ->  qk) 

E^*  =  L  (4) 

k 

Given  an  initial  state  and  the  next  symbol  from  the  original 
process,  only  certain  successor  states  are  accessible.  This  is 
represented  as  the  allowed  state  transitions  resulting  from  a 
single  symbol.  Note  that  7 Tij  =  0  if  S2S3  . . .  sd  A  s\  ■  ■  •  s'D_1 
whenever  q,  =  S1S2  ■  ■  ■  Sjj  and  qj  =  . . .  s'D.  Thus,  for  a 

/i- Markov  machine,  the  stochastic  matrix  II  =  [itij]  becomes  a 
branded  matrix  with  at  most  \A\D  '  1  nonzero  entries.  The  left 
eigenvector  p  corresponding  to  the  unit  eigenvalue  of  II  is  the 
state  probability  vector  under  the  (fast  time  scale)  stationary 
condition  of  the  dynamical  system. 

The  construction  of  a  ZTMarkov  machine  is  fairly  straight¬ 
forward.  Given  D  G  N,  the  states  are  as  defined  in  (2)  and 
(3).  On  a  given  symbol  sequence  S ,  a  window  of  length 
( D  —  1)  is  slided  by  keeping  a  count  of  occurrences  of  se¬ 
quences  Sjj ... .  SiD  SiD+1  and  Sj1  . . .  SiD  which  are  respectively 
denoted  by  N(si1  . . .  SiD  SiD+1)  and  N(st1  . . .  SiD  ).  Note  that  if 
N(si1  . .  .SiD)  =0,  then  the  state  q  =  Sj1  . . .  SiD  G  Q  has  zero 
probability  of  occurrence.  For  N(si1  . . .  SjD  )  A  0,  the  transi¬ 
tions  probabilities  are  then  obtained  by  these  frequency  counts 
as  follows: 

_  -P(s-ii  ■  •  ■  SjD  s)  ^  N(sj1  . . .  SjD  s) 

Jk  P{sil...siD)  ~  N{sil...siD) 

where  the  corresponding  states  are  denoted  by  qj  = 
■  ■  ■  Si.  o  and  qk  =  Sj2  •  •  •  &• 

The  time-series  data  under  the  nominal  condition  generates 
the  state  transition  matrix  II0  that,  in  turn,  is  used  to  obtain  the 
probability  vector  p°,  where  p°  is  the  left  eigenvector  of  II0  cor¬ 
responding  to  the  (unique)  unit  eigenvalue.  Subsequently,  prob¬ 
ability  vectors  {p1 ,  p2, . . .}  are  obtained  at  slow-time  epochs 
{fi,f2>  ■  • .}  based  on  the  respective  time-series  data.  The  behav¬ 
ioral  changes  from  nominal  condition  are  described  as  anomalies 
that  are  characterized  by  a  scalar  called  Anomaly  Measure(Ai). 
The  anomaly  measure  is  based  on  the  following  assumptions. 

1 )  Assumption  1 :  The  evolution  of  damage  is  an  irreversible 
process,  i.e.,  with  zero  probability  of  self  healing.  This 
assumption  implies  the  following  conditions  for  all  time 
t  >  0: 

a)  M  >  0; 

U  #  >  0. 

2)  Assumption  2:  The  damage  accumulation  at  a  slow-time 
epoch  t,  when  the  dynamical  system  has  reached  a  quasi¬ 


steady-state  equilibrium,  is  a  function  of  the  entire  path 
taken  to  reach  that  state. 

Although  the  crack  length  is  traditionally  defined  by  a  straight 
line  joining  the  starting  point  to  the  tip  of  the  crack,  the  actual 
crack  follows  a  complicated  path,  possibly  fractal  in  ductile 
materials,  to  reach  a  particular  point.  Therefore,  assumption  2 
implies  that  the  anomaly  measure  should  be  determined  from  the 
actual  path  traversed  and  not  just  the  end  points.  Accordingly, 
the  path  traversed  also  called  the  scalar- valued  anomaly  measure 
Mk  at  a  slow-time  epoch  tk  is  defined  in  terms  of  a  distance 
function  d(-,  •)  as 

k 

Mk  =  M(tk)  =  ^2d(pl  .p’-1)  (6) 

;=i 

d(x>y)  =  -%l“j  (7) 

where  the  exponent  a  G  [1,  00)  depends  on  the  desired  sensi¬ 
tivity  to  small  deviations.  Small  changes  in  the  signal  profile, 
which  might  be  due  to  spurious  fluctuations  in  the  signal,  can  be 
suppressed  with  large  values  of  a.  Therefore,  the  choice  of  a  is  a 
tradeoff  between  suppression  of  small  spurious  fluctuations  due 
to  noise  and  those  due  to  the  actual  changes  in  the  signal  profile 
resulting  from  damage  growth.  In  this  paper,  the  exponent  is 
chosen  to  be  a  =  2.0,  which  implies  that  d(-,  •)  is  the  standard 
Euclidean  distance.  The  distance  traveled  is  calculated  from  the 
starting  point  (p°  =  uniform  distribution  obtained  with  maxi¬ 
mum  entropy  partitioning)  to  the  current  time  epoch  along  the 
evolution  of  the  probability  vector  (see  Fig.  2). 

VI.  Existing  Pattern-Recognition  Techniques 

This  section  briefly  describes  the  following  three  pattern- 
recognition  techniques,  which  use  time-series  data  as  inputs, 
for  comparison  with  the  symbolic-time-series-based  anomaly 
detection  method: 

1)  PC  A; 

2)  MLPNN; 

3)  RBFNN. 

While  details  of  these  methods  are  provided  in  a  previous 
publication  [  15],  they  are  briefly  described  in  this  paper  for  the 
sake  of  completeness. 

A.  PCA 

Feature  extraction  methods  in  statistical  pattern  recognition 
determine  an  appropriate  subspace  of  dimensionality  q  G  N, 
where  N  is  the  set  of  positive  integers,  using  either  linear  or 
nonlinear  methods  in  the  original  feature  space  of  dimension¬ 
ality  n(q  <  n).  The  best-known  linear  feature  extractor  relies 
on  the  PCA  or  Karhunen-Loeve  expansion  [13].  The  eigenvec¬ 
tors  of  the  (n  x  n)  (positive  semi-definite)  covariance  matrix  of 
the  time-series  data,  corresponding  to  the  q  largest  eigenvalues, 
form  the  n-dimensional  patterns.  The  linear  transformation  is 
defined  as 


Y  =  HX 


(8) 
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where  X  is  the  given  (n  x  d)  pattern  matrix,  made  of  n 
row  vectors;  H  is  the  q  x  n  linear  transformation  matrix  whose 
rows  represent  q  feature  vectors  of  dimension  n ;  and  Y  is  the  de¬ 
rived  d  x  q  pattern  matrix.  Since  the  PCA  method  uses  the  most 
expressive  features  (e.g.,  eigenvectors  with  the  largest  eigenval¬ 
ues),  it  effectively  approximates  the  data  by  a  linear  subspace 
using  the  mean  squared  error  criterion. 

To  detect  growth  in  anomaly  from  time-series  data,  PCA  is 
performed  for  dimensionality  reduction.  If  the  time  response 
of  an  appropriate  process  variable  y(t)  is  sampled  to  generate 
a  time-series  sequence  yk ,  then  data  samples  of  large  enough 
length  (£  =  dn)  can  be  used  to  capture  the  dynamical  character¬ 
istics  of  the  observed  process.  The  length  t  of  time-series  data 
is  partitioned  into  d  subsections,  each  being  of  length  n  =  i/d, 
where  d>n.  The  resulting  (d  x  n)  data  matrix  is  processed  to 
generate  the  (n  x  n)  covariance  matrix  that  is  positive-definite 
or  positive-semidefinite  real-symmetric.  The  next  step  is  to  com¬ 
pute  the  orthonormal  eigenvectors  v1,  v2, . . . ,  vn  and  the  cor¬ 
responding  eigenvalues  Ai,  A2, . . . ,  A„  that  are  arranged  in  de¬ 
creasing  orders  of  magnitude.  The  eigenvectors  associated  with 
the  first  (i.e.,  largest)  q  eigenvalues  are  chosen  as  the  feature 
vectors  such  that 

A, 

—  <  V  (9) 

where  the  threshold  ij  <Cl  is  a  positive  real  close  to  0.  The 
resulting  pattern  is  the  matrix,  consisting  of  the  feature  vectors 
as  columns 

M  =  (\/Aivi  •  •  •  ■  (10) 

The  above  steps  are  executed  for  time-series  data  under  the 
nominal  (stationary)  condition  to  obtain  Mq.  Then,  these  steps 
are  repeated  at  subsequent  slow-time  epochs  {fi,  t-2, . . .}  as  the 
(possible)  anomaly  progresses  using  the  same  values  of  param¬ 
eters  t,  d,  n,  and  </,  used  under  the  nominal  condition,  to  obtain 
the  respective  pattern  matrices  Mi,  M2, . . ..  The  anomaly  mea¬ 
sures  at  slow-time  epochs  {fi,  <2,  ■  •  •}  are  obtained  as 

Mk  =  d 

where  d(- ,  ■ )  is  an  appropriately  defined  path-dependent  distance 
function  as  given  by  (6). 

It  should  be  noted  that  different  metrics  may  be  used  as 
anomaly  measures  as  stated  in  [6].  One  may  choose  the  metric 
that  yields  the  most  satisfactory  result  for  the  specified  purpose. 
Along  this  line,  different  metrics  could  be  chosen  for  other 
pattern-recognition  techniques. 


weights.  A  set  of  inputs  is  passed  forward  through  the  network, 
yielding  trial  outputs  that  are  then  compared  to  the  target  outputs 
to  obtain  the  error  (i.e.,  the  deviation  of  the  trial  output  from  the 
target  output).  The  network  parameters  (i.e.,  synapsis  weights 
and  biases)  are  adjusted  until  the  error  is  within  specified  limits. 
If  the  specified  bound  is  exceeded,  the  error  is  passed  backwards 
through  the  net  and  the  training  algorithm  adjusts  the  synapsis 
weights. 

The  back-propagation  algorithm  has  been  used  in  this  paper. 
The  simplest  implementation  of  back-propagation  learning  up¬ 
dates  the  network  weights  and  biases  in  the  direction  in  which 
the  performance  function  decreases  most  rapidly.  The  mean 
square  error  criterion  is  adopted  in  the  recursive  algorithm  to 
update  the  weight  vectors  {wk}  as  follows: 


wn+i  =  w„  -  a„gn  (ID 

where  g„  is  the  gradient  and  an  is  the  learning  rate. 

Different  layers  in  MLPNN  may  contain  different  numbers 
of  neurons.  Time-series  signals  enter  into  the  input  layer  nodes, 
progress  forward  through  the  hidden  layers,  and  finally  emerge 
from  the  output  layer.  Each  node  i  at  a  given  layer  k  receives  a 
signal  from  all  nodes  j  in  its  preceding  layer  ( k  —  1)  through  a 
synapsis  of  weight  id*,  and  the  process  is  carried  onto  the  nodes 
in  the  following  layer  (k  +  1).  The  weighted  sum  of  signals 
Xj-1  from  all  nodes  j  of  the  layer  (k  —  1)  together  with  a  bias 
wf0  produces  the  excitation  zf  that,  in  turn,  is  passed  through 
a  nonlinear  activation  function  f  to  generate  the  output  x\  from 
the  node  i  at  layer  k.  This  is  mathematically  expressed  as 


Zi=J2WijXJ  1+Wi0 
.1 

(12) 

'  ^ 

II 

-Sfi  •« 

(13) 

Various  choices  for  the  activation  function  /  are  possible;  the 
hyperbolic  tangent  function  f(x)  =  tarili(x)  has  been  adopted 
in  this  paper. 

For  anomaly  detection,  the  MLPNN  is  trained  by  setting  a  set 
of  N  input  vectors,  each  of  dimension  L  and  a  specified  target 
output  vector  r  of  dimension  q.  This  implies  that  the  input  layer 
has  £  neurons  and  the  output  layer  has  q  neurons.  If  the  time- 
series  data  are  obtained  from  an  ergodic  process,  then  a  data 
set  of  length  N.£  can  be  segmented  into  N  vectors  of  length  l 
to  construct  the  input  and  target  pattern  matrices,  V .  The  input 
pattern  matrix  V  £  R£xJV  is  obtained  from  the  N  input  vectors 
as 


B.  MLPNN  for  Anomaly  Detection 

The  MLPNN  is  the  most  commonly  used  family  of  feed¬ 
forward  neural  networks  for  pattern  classification  tasks  [13]. 
The  MLPNN  is  a  collection  of  connected  processing  elements 
called  nodes  or  neurons  [14],  [21],  Its  structure  is  fixed  by 
choosing  the  number  of  layers  as  well  as  the  (possibly  different) 
number  of  neurons  in  each  layer.  The  MLPNN  is  trained  based 
on  the  information  contained  in  a  given  set  of  inputs  and  target 
outputs.  The  training  phase  includes  modeling  of  the  input- 
output  system  architecture  and  identification  of  the  synapsis 


V  =  [p1p2---pN]  (14) 

where  pJ'  =  [y^-i)e+iV(k-i)e+2  ■  ■■yk()T  and  each  yk  is  a  sam¬ 
ple  from  the  ensemble  of  the  time-series  data.  The  corresponding 
output  matrix  O  is  the  output  of  the  trained  MLPNN  under  the 
input  pattern  V 

0=  [oV  •••(>"]  (15) 

where  o*  £  R9  is  the  output  of  the  trained  MLPNN  under  the 
input  pfc  £  Mf .  The  performance  vector  u  £  RR  is  obtained  as 
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the  average  of  the  N  outputs 

1  N 

<16> 

k= 1 

The  time-series  data  under  the  nominal  condition  generates 
the  input  pattern  matrix  Vo  that,  in  turn,  is  used  to  train  the 
MLPNN  with  respect  to  a  target  output  vector  r.  The  resulting 
output  of  the  trained  MLPNN  with  Vo  as  the  input  is  Oq  and 
the  performance  vector  is  Uq,  Subsequently,  input  pattern  matri¬ 
ces  {'Pi,  7^2)  •  •  •}  are  obtained  at  slow-time  epochs  {t\,t2,  ■  ■  •} 
and  corresponding  output  matrices  of  the  trained  MLPNN  are 
{Oi,  C>2,  ■  ■  •},  which  yield  the  respective  performance  vec¬ 
tors  {ui,  U2, . . The  anomaly  measures  at  slow-time  epochs 
{fi, f2,  •  •  •}  are  obtained  as 

Mk  =  d{uk,u0) 

where  d(-,  •)  is  an  appropriately  defined  path-dependent  distance 
function  as  given  by  (6). 

C.  RBFNN  for  Anomaly  Detection 

The  RBFNN  is  a  commonly  used  tool  for  pattern  identifica¬ 
tion  [  14],  where  the  activation  of  a  hidden  unit  is  determined  by 
the  distance  between  the  input  vector  and  the  prototype  vector; 
the  RBFNN  is  essentially  a  nearest-neighbor  type  of  classifier. 
A  radial  basis  function  has  the  following  structure: 

where  the  exponent  parameter  a  €  (0,  oo);  and  /. i  and  9a  are 
the  center  and  nth  central  moment  of  the  data  set,  respectively. 
For  a  =  2,  /(•)  becomes  Gaussian,  which  is  the  typical  radial 
basis  function  used  in  the  neural  network  literature.  To  per¬ 
form  anomaly  detection,  the  first  task  is  to  obtain  the  sampled 
time-series  data  when  the  dynamical  system  is  in  the  nominal 
condition  and  then  the  mean  fi  and  the  central  moment  0a  are 
calculated  as 

N  N 

and  Sa  =  —  -/x|“.  (18) 

fc= 1  fc= 1 

The  distance  between  any  vector  y  and  the  center  ji  is  obtained 
as  d(y,n)  =  |  y(n)  —  /x|“)«  .  Following  (17),  the  radial  ba¬ 

sis  function  at  the  nominal  condition  is  /0  =  f(y)-  Under  all 
conditions  including  anomalous  ones,  the  parameters  /j  and  9 
are  kept  fixed.  However,  at  slow-time  epochs  {ti,t2,  ■  ■  •},  the 
radial  basis  functions  {/i,  /2,  •  •  •}  are  evaluated  from  the  data 
sets  under  the  (possibly  anomalous)  conditions.  The  anomaly 
measure  at  the  epoch  tk  in  the  slow  time  scale  is  obtained  as  a 
distance  from  the  nominal  condition  and  is  given  by 

Mk  =  d(fo,  fk) 

where  df,  •)  is  an  appropriately  defined  path-dependent  distance 
function  as  given  by  (6). 

VII.  Experimental  Validation 

This  section  makes  a  comparative  assessment  of  the 
/V Markov  machine  method  with  different  pattern-recognition 


techniques  for  anomaly  detection.  Time-series  data,  generated 
from  experiments  on  the  test  apparatus  in  Fig.  1,  have  been 
used  for  this  purpose.  Both  the  vibrators  in  the  test  apparatus 
were  excited  by  a  sinusoidal  input  of  amplitude  0.85  V  and 
frequency  1 1 .39  Hz  (approximately  the  resonance  frequency) 
throughout  the  run  of  each  experiment.  The  time-series  data  of 
the  displacement  sensor  on  Mass  #3,  which  serve  as  an  in¬ 
dicator  of  the  system  performance,  were  collected  from  the 
beginning  of  the  experiments  until  breakage  of  specimens.  The 
ensemble  of  data  were  saved  in  a  total  of  35  files,  with  each 
file  containing  a  minute  of  sensor  time-series  data.  Following 
the  procedure  outlined  in  Sections  IV— VI,  the  anomaly  measure 
was  obtained  from  the  data  at  each  2-min  interval  from  the 
sensor  data  contained  in  each  file.  The  time-series  data  sets 
were  collected  after  the  dynamic  response  attained  the  stationary 
behavior.  The  data  set  at  the  time  epoch  of  2  min  was  taken  as  the 
reference  point  representing  the  nominal  behavior  of  the  dynam¬ 
ical  system.  These  data  sets  were  used  to  compare  the  anomaly 
detection  capability  of  the  symbolic  dynamics  approach  relative 
to  that  of  two  existing  pattern-recognition  techniques:  PCA  and 
Neural  Network.  Since  symbol  generation  from  the  time-series 
data  is  the  crucial  step  in  symbolic-dynamics-based  anomaly 
detection,  this  paper  investigates  two  alternative  approaches — 
SFNN  partitioning  and  WS  partitioning. 

The  following  anomaly  detection  approaches  are  investigated 
by  using  the  same  set  of  time-series  data  generated  from  the 
experiments  on  the  test  apparatus  in  Fig.  1 : 

1)  PCA; 

2)  MLPNN; 

3)  RBFNN; 

4)  D- Markov  machine  with  SFNN  partitioning; 

5)  /A  Markov  machine  with  WS  partitioning. 

The  description  of  how  anomaly  measures  are  calculated 
based  on  the  above  five  techniques  of  pattern  recognition  is 
given  later.  The  fourth  paragraph  addresses  both  SFNN  and  WS 
methods  of  partitioning  in  the  /A Markov  method. 

Following  the  PCA  procedure  described  in  Section  VI-A,  a 
block  of  sampled  time-series  data,  having  length  t  =  12  000, 
is  divided  into  n  =  4  segments  of  length  d  =  3000;  these  seg¬ 
ments  are  arranged  to  form  a  3000  x  4  data  matrix.  The  re¬ 
sulting  4x4  (symmetric  positive-definite)  covariance  matrix 
of  the  data  matrix  yields  a  monotonically  decreasing  set  of 
eigenvalues,  Ai  . . .  A4,  and  the  associated  orthonormal  eigen¬ 
vectors  v1, . . . ,  v4.  At  the  nominal  condition,  three  eigenval¬ 
ues  are  found  to  be  dominant  (i.e.,  q  =  3)  for  a  threshold  of 
77  =  1.0  x  10~3  such  that 

^°=3  Xi  =  0.0001  <  77. 

Ef=iA  i 

The  matrix  M0  in  ( 10)  is  calculated  from  the  data  set  at  the  nom¬ 
inal  condition.  Similarly,  the  matrices  M2,  M4, . . . ,  M32  are 
obtained  corresponding  to  different  time  epochs,  respectively. 
The  anomaly  measures  at  different  time  epochs  are  determined 
according  to  (6)  relative  to  nominal  matrix  Mq. 

Following  the  MLPNN  procedure  described  in  Section  VI-B, 
the  resulting  pattern  matrix  Vo  is  made  of  N  =  100  columns. 
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Each  column,  having  a  length  £  =  25,  is  generated  from  the 
time-series  data  at  nominal  condition  to  train  the  MLPNN  that 
is  chosen  to  have  a  input  layer  (with  30  neurons),  four  hidden 
layers  (with  50  neurons  in  layer  1,  40  neurons  in  layer  2,  30 
neurons  in  layer  3,  and  40  neurons  in  layer  4),  and  the  output 
layer  (with  5  neurons).  This  structure  of  the  MLPNN  yields 
very  good  convergence  for  the  data  sets  under  consideration. 
The  target  corresponding  to  each  input  pattern  vector  is  cho¬ 
sen  to  be  5  x  1  zero  vector.  The  MLPNN  is  trained  with  the 
nominal  data  set  and  the  gradient  descent  back-propagation  al¬ 
gorithm  has  been  used  for  network  training  with  an  allowable 
performance  mean  square  error  of  1.0  x  1CL5.  The  input  pattern 
matrices,  Vo ,  V2,  V4, . . . ,  V32 ,  each  of  dimension  (25  x  100), 
are  then  generated  from  the  anomalous  data  sets  at  various  time 
epochs  to  excite  the  trained  network.  The  resulting  output  ma¬ 
trices  of  the  trained  MLPNN  are  Oq  .  O2  ,  O4 , . . . ,  O32,  which 
yield  the  respective  performance  vectors,  Uo,  u2, 114, . . . ,  u32. 
The  anomaly  measures  at  different  time  epochs  are  determined 
according  to  (6). 

Following  the  RBFNN  procedure  described  in  Section  VI-C, 
the  length  of  the  sampled  time-series  data  is  chosen  to  be  N  = 
12  000.  The  exponent  a  for  standard  RBF  is  usually  chosen  to 
be  2,  for  improved  anomaly  measure  sensitivity.  An  estimate  of 
the  parameters  //  and  0,,  are  obtained  according  to  (18)  based  on 
the  data  under  the  nominal  condition,  which  yields  the  requisite 
radial  basis  function  /o  following  (17).  The  anomaly  measures 
at  different  time  epochs  are  determined  according  to  (6). 

Based  on  the  time-series  data  of  the  nominal  condition,  the 
first  step  in  the  D-Markov  machine  method  is  to  find  a  parti¬ 
tion  for  symbol  sequence  generation.  The  partitioning  methods, 
SFNN  and  WS,  described  in  Section  IV,  have  been  investigated 
for  efficacy  of  anomaly  detection.  For  the  given  stimulus  of  this 
experiment,  partitioning  of  the  phase  space/WS  must  remain 
invariant  at  all  epochs  of  the  slow  time  scale.  Absolute  values  of 
the  wavelet  scale  series  data  (see  Section  IV)  were  used  to  gen¬ 
erate  the  partition  because  of  the  symmetry  of  the  data  sets  about 
their  mean.  The  finite-state  machine  constructed  with  the  choice 
of  the  parameters  |*4|  =  8  and  D  -  1  has  only  eight  states  and 
it  was  able  to  capture  early  anomalies.  Increasing  the  value  of 
\A\  further  did  not  improve  the  results  and  increasing  the  value 
of  depth  D  created  a  large  number  of  states  of  the  finite-state 
machine,  many  of  them  having  very  small  or  zero  probabili¬ 
ties.  Hence,  the  value  of  D  =  1  was  used  for  construction  of 
the  D-Markov  machine  for  data  at  all  time  epochs.  Follow¬ 
ing  the  procedure  described  in  Section  V,  the  state  machines 
are  constructed  to  generate  the  connection  matrix  n  =  [7 Tjk] 
and  the  state  probability  vector  p  for  each  slow  time  epoch. 
The  state  machines  were  constructed  with  a  symbol  alphabet  of 
cardinality  8. 

A.  Comparison  of  Anomaly  Detection  Methods 

The  anomaly  detection  method  is  validated  on  the  labora¬ 
tory  test  apparatus  in  Fig.  1.  Test  runs  are  planned  for  exoge¬ 
nous  stimuli  at  one  of  the  natural  frequencies  of  the  mechanical 
structure  to  cause  fatigue  failure  within  reasonable  length  of 
time.  Using  the  same  set  of  time-series  data  generated  from  the 


Fig.  3.  Performance  comparison  of  anomaly  detection  methods. 


above  experiments,  the  live  plots  in  Fig.  3  compare  the  anomaly 
measures  obtained  by  using  the  aforementioned  five  anomaly 
detection  approaches  WS,  SFNN,  PCA,  MLPNN,  and  RBFNN, 
for  the  first  16  files  (i.e.,  up  to  ~32  min)  when  the  service  life 
of  the  test  specimen  has  expired,  i.e.,  the  specimen  is  about  to 
break.  (Note:  The  estimated  service  life  of  the  specimen  un¬ 
der  this  load  excitation  is  ~30  min.)  The  nominal  condition  is 
chosen  at  the  time  epoch  of  2  min  to  ensure  that  all  transients 
have  decayed.  Each  of  the  five  methods  shows  a  sharp  rise  in  the 
anomaly  measure  at  about  22  min,  which  is  indicative  of  the  tran¬ 
sition  from  the  crack-incubation  to  the  crack-propagation  phase 
in  fatigue  damage.  The  symbolic  time-series-based  anomaly 
detection  with  both  WS  and  SFNN  give  better  performance 
than  the  three  other  existing  pattern-recognition  tools  and  are 
responsive  to  progressive  damage  in  the  short-crack  regime 
till  the  time  epoch  of  ~22  min.  At  this  time,  cracks  begin  to 
form,  indicating  the  transition  from  crack  incubation  to  crack 
initiation. 

Physical  modeling  of  fatigue  damage  in  the  short  crack  re¬ 
gion  has  not  been  dealt  with  significantly  in  technical  literature. 
Fig.  3  shows  that  symbolic  time-series  analysis  under  both  WS 
and  SFNN  partitioning  is  able  to  capture  the  progressive  accu¬ 
mulation  of  fatigue  damage  in  the  crack-initiation  phase  while 
the  performance  of  PCA  and  RBFNN  is  somewhat  inferior  and 
MLPNN  is  the  worst.  This  is  indicated  by  the  magnitude  of 
the  anomaly  measure  as  well  as  the  change  in  slope  starting  at 
~15  min.  Slope  of  the  anomaly  measure  curve  is  representa¬ 
tive  of  the  rate  of  damage  progression.  The  distributed  nonlin¬ 
earities  in  MLPNN  may  not  be  specifically  suited  to  capture 
these  small  parameter  perturbations  in  the  largely  linear  behav¬ 
ior  of  the  dynamic  response  of  the  vibrating  structures.  The 
WS  partitioning  shows  higher  anomaly  measure  as  compared 
to  PCA.  The  rationale  is  that  the  PCA  method  is  dependent 
on  eigenvalues  and  eigenvectors  of  the  covariance  matrix  that 
is  sensitive  to  measurement  noise  in  the  data  acquisition  pro¬ 
cess.  In  contrast,  the  symbolic  time-series  analysis,  with  both 
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WS  and  SFNN  partitioning,  is  much  less  sensitive  to  (zero- 
mean)  measurement  noise  because  of  the  inherent  averaging 
due  to  repeated  path  traversing  in  the  finite-state  machine.  All 
five  anomaly  detection  methods  exhibit  a  gradual  increase  in 
anomaly  measure  from  the  beginning  up  to  the  time  epoch  of 
~22  min,  when  the  transition  takes  place  from  the  short-  to  the 
long-crack  regime.  RBFNN,  MLPNN,  and  PCA  exhibit  good 
performance  in  the  long-crack  regime  in  the  24— 28-min  interval. 
The  specimen  breaks  at  ~28  min. 


VIII.  Summary,  Conclusion,  and  Future  Work 

This  paper  experimentally  validates  a  novel  tool  using  WS 
partitioning  [6],  [10]  for  the  detection  of  evolving  anomalies 
in  dynamical  systems  and  compares  it  with  existing  pattern- 
recognition  techniques:  PCA,  MLPNN,  and  RBFNN.  It  is  as¬ 
sumed  that  the  unforced  dynamical  system  (i.e.,  in  the  absence 
of  external  stimuli)  is  stationary  at  the  fast  time  scale  and  that 
any  nonstationary  behavior  is  observable  only  on  the  slow  time 
scale.  Then,  an  HMM  [6],  [22],  is  constructed  from  the  symbol 
sequences  in  the  form  of  a  probabilistic  finite-state  automaton. 
Anomaly  measures  at  different  slow-time  epochs  are  obtained 
in  real  time  as  the  distance  between  the  state  probability  distri¬ 
bution  at  that  epoch  and  the  state  probability  distribution  at  the 
nominal  condition  incorporating  the  path  traversed  by  the  state 
probability  vector.  In  this  way,  the  anomaly  measure  quantifies 
damage  growth  relative  to  the  nominal  condition. 

The  following  observations  are  made  from  the  analysis  of 
experimental  data. 

1 )  The  results  of  symbolic  time-series  analysis  on  the  labo¬ 
ratory  apparatus  show  that  the  evolution  of  fatigue  crack 
damage  is  detected  in  advance  of  component  failure.  This 
is  of  paramount  importance  to  health  and  usage  monitor¬ 
ing  of  machinery  operations  as  it  provides  ample  time  to 
take  remedial  control  actions  for  life  extension  [12]  and 
reliability  enhancement. 

2)  The  symbolic  time-series  analysis,  with  WS  and  SFNN 
partitioning,  is  more  effective  for  the  detection  of  evolving 
anomalies  in  the  crack  initiation  regime  than  the  PCA, 
MLPNN,  and  RBFNN  methods. 

A  major  conclusion,  based  on  this  specific  experimental  inves¬ 
tigation,  is  that  symbolic  time  series,  along  with  the  stimulus- 
response  methodology  and  having  a  vector  representation  of 
anomaly,  is  effective  for  the  detection  of  small  anomalies  due  to 
fatigue  damage  in  ductile  alloy  materials. 

Further  theoretical  and  experimental  research  is  recom¬ 
mended  in  the  following  areas: 

1)  theoretical  research,  supported  by  experimental  valida¬ 
tion,  in  phase-space  and  wavelet-space  partitioning  for 
generation  of  symbol  sequences  from  time-series  data; 

2)  validation  of  the  symbolic  time-series  method  for  the  de¬ 
tection  of  fatigue  damage  in  different  materials  with  vari¬ 
ous  geometries  and  loading  conditions; 

3)  understanding  deeper  aspects  of  fatigue  crack  initiation 
using  advanced  sensing  technology  such  as  ultrasonics 
and  electromagnetics. 
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